Virtual reality system locomotion interface utilizing a pressure-sensing mat attached to movable base structure

ABSTRACT

A virtual reality system transposes a user&#39;s position and movement in real space to virtual space. The virtual reality system includes a locomotion interface that outputs signals indicative of a user&#39;s position in real space. The locomotion interface includes a pressure-sensing mat having a base layer, a plurality of pressure sensing elements formed over the base layer, a top layer formed over the plurality of pressure-sensing elements, and an input interface formed between the base layer and the top layer. The locomotion interface further includes a base around which the pressure sensing mat is disposed, the base structure being fixed in a first position but freely moveable in a second position. The plurality of pressure sensing elements output a signal indicative of pressure applied to the top layer. A virtual reality processor uses the signals output by the locomotion interface to produce an output indicative of the user&#39;s position in the virtual space corresponding to the user&#39;s position and movement in the real space. A display uses the output from the virtual reality processor to produce an image of the virtual space.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority as a continuation-in-part of U.S.patent application Ser. No. 10/417,095 entitled VIRTUAL REALITY SYSTEMLOCOMOTION INTERFACE UTILIZING A PRESSURE-SENSING MAT filed on Apr. 17,2002, and as a continuation-in-part of U.S. patent application Ser. No.10/045,052 entitled VIRTUAL REALITY SYSTEM LOCOMOTION INTERFACEUTILIZING A PRESSURE-SENSING MAT filed on Jan. 15, 2002, which claimsthe benefit of U.S. Provisional Application Ser. No. 60/306,854, filedJul. 23, 2001. The content of each of the above identified applicationsis incorporated herein by reference in its entirety.

BACKGROUND OF THE INVENTION

1. Field of Invention

This invention relates to virtual reality systems that can be used tofully immerse a user in virtual space.

2. Description of Related Art

Virtual reality is a computer-generated environment in which a user isimmersed. Actions of the user are translated by a computer into inputsthat effect the virtual environment (VE). Virtual reality systems maystimulate naturally occurring senses, such as sight, sound, touch andmovement, so that a user can navigate through a virtual environment asif in the real world.

A major challenge to virtual reality system designers is to design avirtual reality system that allows natural human locomotion. Previousvirtual reality systems that allow the user to move naturally requirecomplex and expensive equipment. Other virtual reality systems abandonthe concept of natural human locomotion, using simple hardware thatallow the user to navigate through the virtual environment withartificial gestures, such as flying in the virtual space in thedirection the user's finger is pointing.

Known virtual reality systems include treadmill devices that track theuser's movement on the treadmill. Such a device is disclosed in U.S.Pat. No. 5,562,572 to Carmein. Although these treadmill devices allowmovement in the user's upright position, they do not allow movement inthe user's prone position. They also cannot sense whether the user is inthe standing, crawling or prone position. Further, these treadmilldevices are often mechanically complicated, and are thus encumbered bythe inherent lag times and momentum problems associated with movingmechanical masses.

Other known virtual reality systems allow the user to move in the proneposition, but sacrifice natural motion. For example, one known deviceincludes a simple foot-pedal interface, similar to the accelerator of anautomobile. The foot-pedal allows the user to move forward or backward,depending on where the user presses the foot-pedal. In this system, theuser always moves toward the center of the field of view, and the fieldof view is rotated if the user turns his head past a certain angle.Although this system allows a user to navigate from any posture, theuser must be in constant contact with the foot-pedal to navigate. Italso does not enable the user to move naturally.

SUMMARY OF THE INVENTION

In various exemplary embodiments, the virtual reality system accordingto one aspect of this invention includes a pressure-sensing mat thatoutputs signals indicative of a user's position in real space. A virtualreality processor uses the signals output by the pressure-sensing mat toproduce an output indicative of the virtual space corresponding to theuser's position and movement in real space. A display device uses theoutput from the virtual reality processor to allow the user to be fullyimmersed in the virtual space.

In various exemplary embodiments, the pressure sensing mat includes abase layer, a plurality of pressure sensing elements formed over thebase layer, and a top layer formed over the plurality ofpressure-sensing elements. The plurality of pressure sensing elementsoutput a signal indicative of pressure applied to the top layer.

In various exemplary embodiments, the pressure sensing mat having a baselayer is part of a large enclosed structure that is fixed in a firstposition but rotatebly movable in a second position such that it movesunder a user's body as the user moves in virtual space.

In various exemplary embodiments, the pressure sensing mat having a baselayer attached to an enclosed structure rotatebly movable about a secondposition allows the user to move in one or more of at least a forwardmoving position, a backward moving position, a left side movingposition, a right side moving position and a position therebetween.

This invention provides a virtual reality system that has a simpledesign and that allows a user to move naturally in any direction fromany posture (e.g., standing, crawling, prone). The virtual realitysystem according to this invention has many advantages over previousvirtual reality systems. The enhanced flexibility of the variousexemplary embodiments of the system according to this invention allows auser to move forward, backward, or sideways from a prone, crawling orstanding position. Thus, the virtual reality system according to thisinvention has many applications, such as, for example, enhanced militarytraining, realistic video game environments, and a broad range ofmedical and therapeutic applications.

These and other features and advantages of this invention are describedin, or are apparent from, the following detailed description of variousexemplary embodiments of the systems and methods according to thisinvention.

BRIEF DESCRIPTION OF THE DRAWINGS

Various exemplary embodiments of this invention will be described indetail, with reference to the following figures, wherein:

FIG. 1 illustrates one exemplary embodiment of a virtual reality systemaccording to this invention;

FIG. 2 illustrates one exemplary embodiment of the pressure sensing mataccording to this invention;

FIG. 3 shows one exemplary embodiment of a pressure sensitive resistorusable with the various exemplary embodiments of the virtual realitysystem according to this invention;

FIG. 4 illustrates the equivalent circuit of the pressure sensing mataccording to this invention;

FIG. 5 is a block diagram of an exemplary embodiment of the virtualreality processor according to this invention; and

FIG. 6 illustrates one exemplary embodiment of a movable pressuresensing mat enclosure structure according to this invention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

FIG. 1 illustrates one exemplary embodiment of a virtual reality systemaccording to this invention. The virtual reality system 1 includes apressure sensing mat 100, a virtual reality (VR) processor 200, and adisplay 400. It should be appreciated that the various exemplaryembodiments of the virtual reality system according to this inventioncan have any number and configuration of components that use a pressuresensing mat to sense the user's movement in order to generate a virtualenvironment.

FIG. 2 illustrates one exemplary embodiment of the pressure sensing mat100 according to this invention. The pressure sensing mat 100 includes asemi-rigid base layer 120. Any suitable material can be used for thebase layer 120, such as, for example, plastic, hardwood, andpolycarbonate (lexan). A grid 140 (i.e., a two-dimensional array) ofpressure sensing elements 150-1 to 150-n is formed over the base layer120. A top layer 160 is formed over the grid 140. Any suitable layer canbe used for the top layer 160, such as, for example, rubber, naturalrubber, buna's rubber, and fabric reinforced negro rubber, is preferred.

In various exemplary embodiments, the pressure sensing mat 100 mayinclude a pressure sensing mat input interface 170 having one or morecommunication interfaces 175 to communicate with the VR processor 200and/or display 400. The pressure sensing mat input interface 170 canreceive analog voltage signals from the pressure sensing elements 150-1to 150-n. The pressure sensing mat input interface 170 can include ananalog to digital converter that converts the analog voltage signals todigital signals. The pressure sensing mat input interface 170 can usethe one or more communication interfaces 175, such as, for example, theradio-frequency communication interfaces 175, to transmit the digitalsignals from the pressure sensing mat to the VR processor and/or thedisplay 400.

In various exemplary embodiments, the radio-frequency communicationinterface 175 includes at least one of Bluetoothe®—type communicationinterface, a short range FM transmitter, a 802.11 transmitter, an 802.11g transmitter, and the like.

In an exemplary embodiment, pressure-sensing mat input interface 170 andcommunication interface 175 are embedded within the pressure sensing matlayers 120, 160. The one or more communication interfaces 175 mayinclude devices employing any type of communication modalities, such as,for example, electrical and electromagnetic communication modalities. Ina preferred embodiment, the one or more communication interfaces 175 isa radio-frequency communication interface.

The pressure sensing elements 150-1 to 150-n of the grid 140 detect thepressure applied to fixed points on the top layer 160 of the pressuresensing mat 100. Any suitable pressure sensing device can be used forthe pressure sensing elements 150-1 to 150-n, such as, for example,electromechanical pressure sensors. In general, any known or laterdiscovered pressure sensing device can be used for the pressure sensingelements 150-1 to 150-n.

In the exemplary embodiment shown in FIG. 1, the pressure sensingelements 150-1 to 150-n include force sensitive resistors. As is knownin the art, force sensitive resistors include elements that act assimple voltage dividers. FIG. 3 shows one exemplary embodiment of apressure sensitive resistor 180 usable with the various exemplaryembodiments of the virtual reality system according to this invention.The pressure sensing elements 150-1 to 150-n include correspondingpressure sensitive resistors 180-1 to 180-n. Each pressure sensitiveresistor 180 includes an upper film 181, a lower film 182, a firstelectrode pattern 183 formed over the lower film 182, a second electrodepattern 184 formed over the upper film 181 so as to oppose the electrodepattern 183, and a pressure-sensitive conductor 185 formed over thesecond electrode pattern 184. When the upper film 181 is pressed, thepressure sensitive conductor 185 is compressed between the first andsecond electrode patterns. As is known in the art, the resistance of thepressure sensitive conductor 185 is lowered when compressed.Accordingly, voltage output of the pressure sensitive resistor 180 willvary with applied pressure. For more details of a pressure-sensitiveresistor, see U.S. Pat. No. 5,948,990, the disclosure of which isincorporated herein by reference.

FIG. 4 illustrates the equivalent circuit of the pressure sensing mat100. The voltage outputs Vout-1 to Vout-n correspond to respectivepressure sensing elements 150-1 to 150-n that make up the grid 140. Auser applies pressure to points on the pressure sensing mat 100 as theuser navigates through the virtual reality environment. The appliedpressures alter the resistance of the pressure sensitive resistors 180-1to 180-n, and thus the voltage output of each of the correspondingpressure sensing elements 150-1 to 150-n varies as the user moves. Thegrid 140 produces a voltage output that can be analyzed to generate apattern that shadows the user's movements in the virtual space.

FIG. 5 is a block diagram of an exemplary embodiment of the virtualreality processor 200. The virtual reality processor includes acontroller 210, a memory 220 (including RAM and ROM, for example), apattern generation device 230, a motion identification device 240, avirtual environment rendering device 250, an input interface 260, and anoutput interface 270. The input interface may include a communicationinterface 265, for example a radio-frequency communication interface265, similar to the pressure sensing mat communication interface 165discussed above.

The controller 210 interfaces with the other components 220-270 using acontrol/data bus 280. Although the exemplary virtual reality processor200 uses a bussed architecture, it should be appreciated that theexemplary virtual reality processor 200 can use any known or laterdeveloped architectures, including ASIC, a programmed general purposecomputer, discrete logic devices, etc.

Under control of the controller 210, the input interface 260 can receivedigital or analog voltage signals from the pressure sensing elements150-1 to 150-n using the communication interface 265. In cases where theinput interface 260 receives analog voltage signals, the input interface260 can include an analog to digital converter that converts the analogvoltage signals to digital signals. The input interface 260 can inputthe digital signals to the memory 22.0 for storage. Alternatively, theinput interface 260 can use the radio-frequency communication interface265 to input the digital signals into memory 220

Next, the controller 210 can provide the digital signals stored in thememory 220 to the pattern generation device 230. The pattern generationdevice 230 samples the digital signals stored in the memory 220 atregular intervals and generates a pattern based on the digital signalsat the regular intervals. The patterns generated by the patterngeneration device 230 represent various positions of the user on thepressure sensing mat 100.

The controller 210 transfers the patterns generated by the patterngeneration device 230 to the motion identification device 240. Themotion identification device 240 can include a pattern recognitiondevice (not shown) that identifies a given pattern with a correspondingposition of the user. The pattern recognition device can identify apattern by comparing the pattern with a database of patterns stored inthe memory 220. The pattern recognition device can also recognize thepattern based on the size, shape and/or pressure distribution of thepattern. For example, if the pattern is larger than a predeterminedthreshold size, the pattern recognition device will recognize thepattern as a “prone user position” pattern. Similarly, if the matoutputs signals indicative of two patterns of a similar size thatalternately move, the processor determines that the user is upright(e.g., walking, running or standing (if the two patterns do not move)).If more than two smaller moving patterns are detected, the user isdetermined to be crawling. The patterns stored in the memory 220 canprovide examples for a neural network to learn how to identify differentpatterns.

Based on the posture and directional information determined by theprocessor, the virtual environment (i.e., the displaying image) isappropriately altered.

A series of user positions identified by the pattern recognition devicecan be stored in the memory 220 during fixed intervals as the usernavigates through the virtual environment. Preferably, the centroid ofeach of the patterns in the series of patterns is tracked as the usermoves on the pressure sensing mat 100. The motion identification device240 can sample the series of user positions at the end of the fixedintervals and identify the motion of the user during the fixed intervalsbased on the series of user positions. The motion includes, for example,direction (forward, backward, left, right, etc.) and speed. The patternsalso can be analyzed to determine the posture (standing, crawling,prone) of the user.

The direction that the user is facing is determined by a sensor that canbe directly attached to the user. In embodiments, the sensor can be amagnetic tracker attached to the user's waist that determines thedirection the waist is facing. The virtual reality system according tothis invention provides significant advantages over known virtualreality systems in that only a single sensor needs to be directlyattached to the user. Thus, the user is relatively free from cumbersomesensor wiring and devices.

The controller 210 can transpose the motion of the user into the virtualenvironment generated by the virtual environment rendering device 250.Data for the virtual environment, including virtual objects, can bestored in the memory 220. The virtual environment rendering device 250can update the virtual environment at given intervals based on the datastored in the memory 220. The virtual environment rendering device 250can update the virtual space each time the user's motion is identified.Thus, as the user moves through the virtual space, the user can effect,and can be effected by, the virtual environment. For example, as theuser navigates through the virtual space, the user's perspective in thevirtual space can change, virtual objects can enter the user's path, andthe user can move virtual objects.

The controller 210 can control the output interface 270 to outputvirtual reality environment data to the display 400. Although thedisplay 400 is shown in FIG. 1 as a head-mounted display, any known orlater discovered display can be used. Preferably, the display providesthe user with the ability to see, hear, smell and/or touch in thevirtual world so that the user is fully immersed in the virtual space.

In various exemplary embodiments, the pressure sensing mat 100 may be aslarge as required to allow the user to move as if the user was in thevirtual space. For example, the pressure sensing mat 100 can be made tocover the floor of a large field or room. Alternatively, if space islimited, the pressure sensing mat 100 can be made smaller, in which casethe user would be required to move in a bounded area or move “in place”.

In a preferred exemplary embodiment, the pressure sensing mat, which mayinclude a base layer, is part of a large enclosed structure. Theenclosed structure is fixed in a first position but rotatebly movable ina second position such that it moves under a user's body as the usermoves in virtual space. In this exemplary embodiment, the pressuresensing mat allows the user to move in one or more of at least a forwardmoving position, a backward moving position, a left side movingposition, a right side moving position and a position therebetween.

FIG. 6 illustrates a cross section of a locomotion interface 300according to an embodiment of this invention including a pressuresensing mat 305 wrapped around a spheroid base 330.

The pressure sensing mat 100 can be arranged on a movable structure,such as, for example in the form of a tread-mill. Various other movablestructures may be employed in addition to the tread-mill-typestructures. As shown in FIG. 6, the pressure sensing mat 305 includes abase layer 310, a grid 315 of pressure sensing elements 320-1 to 320-nformed over the base layer 310, and a top layer 325 formed over the grid315. The pressure sensing mat 305 further includes an input interface370 having one or more communication interfaces 375. The input interface370 together with the one or more communication interfaces 375 may beformed between base layer 310 and top layer 325 of the pressure-sensingmat 305. The pressure sensing mat 305 can be held on to the surface ofthe spheroid base 330 by its own elasticity, thus making the contactbetween the spheroid base 330 and the pressure sensing mat 305relatively frictionless.

The locomotion interface 300 includes a housing 335 that retains thepressure sensing mat 305 and the spheroid base 330. Passive casters 385can be mounted in the housing 335 to allow the pressure sensing mat 305to move freely in all directions in the housing 335.

In this embodiment, the locomotion interface 300 must have thecapability of allowing a user to feel as if he/she can move in alldirections for an “infinite” distance. Thus, the pressure sensing mat305 must be able to move underneath the user as the user “moves” in thevirtual environment. However, although the pressure sensing mat 305moves with little friction on the spheroid base 330, the mass of thepressure sensing mat 305 will not allow the user to propel the pressuresensing mat 305 underneath him/herself as the user moves. Thus, in thisembodiment, the pressure sensing mat 305 is mechanically actuated tomove underneath a user. For example, as shown in FIG. 6, a steerableroller 340 is disposed in the housing 335, and the steerable roller isin frictional contact with the pressure sensing mat 305. The roller 340is steerable about a first axis 350 and a second axis 360. The firstaxis 350 is perpendicular to the second axis 360. A first motor 345powers the roller 340 about the first axis 350, and a second motor 355powers the roller 340 about the second axis 360. The thrust vectorsgenerated by the roller 340 cause the pressure sensing mat 305 to slidearound the spheroid base 330 in all directions.

A sensor (not shown) can be placed above the locomotion interface 300 tosense the direction in which the user is facing. The sensed userdirection can then be used to determine the appropriate thrust vector tobe generated by the roller 340, to thereby move the pressure sensing mat305 underneath the user. For example, if the sensor determines that theuser is facing a first direction, the roller 340 can be controlled tocreate a thrust vector in the first direction to thereby move thepressure sensing mat underneath the user in a second direction oppositethe first direction.

The roller 340, casters 385 and other mechanical parts of the locomotioninterface 300 may interfere with accurate pressure sensing by thepressure sensing mat 305. Further, the pressure sensing mat 305 will bemoving underneath the user as the user “moves” in the virtualenvironment, which will also diminish the accuracy of the pressuresensing. In order to solve these problems, when the user first steps onthe pressure sensing mat 305, an initialization procedure can be done inwhich the users position on the pressure sensing mat 305 is determined.

The virtual reality system 1 can be implemented as software executing ona programmed general purpose computer, a special purpose computer, amicroprocessor or the like.

While the invention has been described with reference to exemplaryembodiments thereof, it is to be understood that the invention is notlimited to the preferred, exemplary embodiments or constructions. To thecontrary, the invention is intended to cover various modifications andequivalent arrangements. In addition, while the various elements of theexemplary embodiments are shown in various combinations andconfigurations, which are exemplary, other combinations andconfigurations, including more, less or only a single element, are alsowithin the spirit and scope of the invention.

1. A locomotion interface that provides input signals, indicative of a user's movement, to a virtual reality system, the locomotion interface comprising: a pressure-sensing mat including a base layer, a plurality of pressure sensing elements formed over the base layer, a top layer formed over the plurality of pressure-sensing elements, and an input interface formed between the base layer and the top layer, wherein the plurality of pressure sensing elements output signals indicative of pressure applied to the top layer; a base structure coupled to the pressure-sensing mat, the base structure being fixed in a first position but freely moveable in a second position; a housing that retains the pressure-sensing mat and the base, said housing further comprising a roller disposed in the housing, the roller being in frictional contact with the pressure-sensing mat; a first motor that rotates the roller about a first axis; and a second motor that rotates the roller about a second axis, the second axis being perpendicular to the first axis, wherein the rotation of the roller generates thrust vectors that move the pressure-sensing mat in all direction.
 2. The locomotion interface of claim 1, wherein the base structure moves the pressure-sensing mat in all directions.
 3. The locomotion interface of claim 1, wherein the pressure-sensing mat allows the user to move in one or more of at least a forward moving position, a backward moving position, a left side moving position, a right side moving position and a position therebetween.
 4. The locomotion interface of claim 1, wherein the input interface comprises one or more radio-frequency communication interfaces.
 5. The locomotion interface of claim 1, further comprising a plurality of casters disposed between the housing and the pressure-sensing mat, the casters allowing the pressure-sensing mat to move in the housing.
 6. The locomotion interface of claim 1, wherein the plurality of pressure-sensing elements make up a grid.
 7. The locomotion interface of claim 1, wherein the plurality of pressure-sensing elements comprise force sensitive resistors.
 8. The locomotion interface of claim 1, wherein the base layer comprises a semi-rigid material.
 9. The locomotion interface of claim 1, wherein the base layer comprises plastic.
 10. The locomotion interface of claim 1, wherein the top layer comprises rubber.
 11. A virtual reality system comprising the locomotion interface of claim
 1. 